This invention relates to improvements in the process of developing new products and services, and the attendant activities of consumer research, market segmentation, design iteration, market testing, and discerning consumer preference trends and attitudes, through direct customer participation.
In the past, many product development organizations relied on a few individuals in their design or marketing departments, or in their executive ranks, when designing new products. These individuals in turn relied on their knowledge of the market and the customer, on their understanding of the technological possibilities, and on their vision, judgment, experience, preferences, prejudices and biases.
More recently, companies have adopted flatter, less hierarchical organizational models, with decision-making responsibility pushed lower through the ranks, and they have embraced a new focus on the “voice of the customer.” This movement was intended to remind them that as designers, they are mere proxies for the ultimate consumer of the goods or services being designed, and that the needs and desires of the customer should be the paramount input to that process.
But the process of going from the voice of the customer to a product or service that reflects it remains fraught with errors and the potential for distortions. The first source of error is in ascertaining the wants and needs of the customer; the second is in the process of translating that input into a decision, product, artifact or service without coloring and distorting it. Practitioners have developed and used several tools and techniques intended to assess the needs of the customer and to translate these needs into a product concept and into engineering requirements.
The tools and instruments that traditionally have been deployed by market researchers range from the highly qualitative methods borrowed from ethnography, such as open-ended interviewing, participant observation, and focus groups, to the highly popular quantitative statistical methods such as survey research and conjoint analysis. Co-pending U.S. application Ser. No. 10/053,353 filed Nov. 9, 2001 titled “METHOD AND APPARATUS FOR DYNAMIC, REAL-TIME MARKET SEGMENTATION”, the disclosure of which is incorporated herein by reference, discloses a novel family of tools that have had very significant commercial success.
During the development of a new product or service, the design organization typically will undertake a number of market research studies. Early on during the project, these may be more qualitative in nature, intended to uncover latent needs, or to develop new ideas for products and services. Later, the research may be more focused, intended to obtain feedback from current or potential customers on certain features or attributes of the proposed product. These could rely on qualitative methods, a focus group for example, as well as quantitative ones, such as surveys or structured serial interviews. One problem with consumer clinics that ask participants for feedback on new products is that potential customers are typically shown, and asked to comment on, only a limited number of alternatives. This is done in order to keep the cognitive demands on the participants at a reasonable level and because the designs shown to the participants are in the form of models or prototypes that are sometimes costly to produce.
Furthermore, consumer clinics assume that people have preexistent preferences that are well-developed and stable. They therefore assume that the attitude that the participants form upon seeing the new product are valid and reflect the attitudes they will have when (and if) the product goes on the market. Yet, it is well-known that in many cases, people's long-term disposition towards a product differs from their initial reaction.
Another problem with clinics and focus groups has to do with the interpersonal dynamics that the situation entails. In general, group dynamics are desirable in the sense that the discussion that takes place between participants is the mechanism for generating data, and the desired output is the active sharing and comparison of the participants' experiences and opinions. Problems arise when one or a few strong individuals end up dominating and biasing the discussion. Another difficulty is finding participants who do not know one another. This is desirable in order to avoid having one participant choose a particular design simply because his or her friend also chose it. This situation arises often when the product or service being designed is targeted at a small group of users, or users who are all members of the same group, for example, designing a benefits package for the employees of one company. Similar problems arise when the potential customers for a product happen to be competitors, and therefore less willing to sit together and share their preferences.
Conjoint analysis is used to assess consumer preference for different choices of products and services. It is a multi-attribute utility or preference measurement technique that explicitly accounts for the subjective tradeoffs people make when deciding among alternatives with multiple features and benefits. In its basic form, conjoint analysis is a decompositional technique: the parameters that measure the importance the decision maker ascribes to the different aspects of the product are derived, using statistical regression techniques, from the decision maker's evaluations of a number of full profile descriptions of the product or service. Conjoint analysis has been used in a wide range of applications, from developing soaps and dietary supplements to improving the appeal of military careers within the Department of Defense.
The first step in conducting a conjoint exercise is to identify the relevant attributes of the product or service in question, and to identify the levels of interest for each attribute. This is typically based on previous experience with similar products, and, on earlier qualitative research such as an open-ended interview or a focus group. As an example, in the case of an automobile study, engine displacement may be one attribute of interest, with 2.0, 2.5, and 3.0 liters the three levels to be tested; and body style may be another attribute, with “sedan” and “coupe” as the levels of interest. Next, a number of full-profile descriptions of potential products, that is, descriptions in which every attribute is represented by a value, usually using a highly fractionated factorial orthogonal design (i.e., only a small fraction of all possible product profiles are used in the test.) These profiles are shown to the respondent, traditionally in the form of prop cards, and the respondent is asked to rank them by order of preference or to rate each of them on an interval scale, for example, from 0-100. The responses then are analyzed using statistical tools such as Ordinary Least Squares regression to estimate the “part-worths” for each of the attribute levels, that is, the contribution of each attribute level to the overall preference level of a profile. Returning to the earlier example, it might turn out that for one particular respondent, a 2.0 liter engine has a part-worth of 0.0, the 2.5 liter a part-worth of 0.5, and so on; the “sedan” body style may have a part worth of 0.0, whereas the “coupe” style may have a value of 0.8. Once the part-worths for individual attributes are obtained in this way, it is then possible to search through all the possible combinations of attribute levels to synthesize the optimal product for that individual, that is, the product that would give him or her the highest possible level of utility, or that he or she would have the strongest intention of buying.
Conjoint analysis studies typically are conducted with more than one individual, and part-worths are typically obtained for a representative sample of consumers. This multi-respondent data can be used for several purposes. One is to identify the product design that would result in the greatest market share for the product development organization, given the attributes of competing products on the market (current and expected; this is known as the “share-of-choices” problem). Another purpose is to identify the product design that would maximize overall consumer utility, that is, the sum of utilities across all the consumers; this is known as the “buyer's welfare” problem. Solving these search problems is a hard computationally; mathematically, these are known as NP-Hard problems, requiring heuristic dynamic programming procedures for their solution. More recently, the adaptive search techniques of Genetic and Evolutionary Computation, more specifically Genetic Algorithms (GAs), have been used more effectively to find solutions to these problems.
Another purpose of collecting conjoint data from a representative group of participants is to identify distinct market segments with different preference profiles. This is done through cluster analysis, a statistical technique for finding subgroups of respondents such that respondents within a subgroup value the different product attributes similarly, but differently from respondents in other subgroups. Once clusters are identified, those that present significant commercial potential can be targeted with specific product designs.
Conjoint analysis has shortcomings. The first is the tediousness of participating in the process as a respondent. Generally, the product designers and marketers, by virtue of their intimate involvement with and knowledge of the product, want to answer a large number of issues and test a large number of attributes. The customers on the other hand are generally less engaged and reluctant to submit to lengthy questionnaires. And even though highly fractionated factorial designs are used, respondents are still typically asked to rate a considerable number of possibilities. For example, in a case where there are 12 product attributes, with four different levels for each attribute, the respondent would face about 35 profiles. That number is often multiplied by a factor of 3 in order to reduce the effect of random errors, resulting in the respondent having to face over 100 questions. The laboriousness of the process often leads to confusion and loss of attention and focus on the part of the respondents, who often end up resorting to heuristics as a shortcut for getting through the questionnaire (several example conjoint exercises can be found on the World Wide Web; see, for example, www.conjointonline.com). For example, instead of properly weighing all the attributes against one another, they only rely on one or two to make their decision, leading to inaccurate results.
More recently, several modifications to conjoint analysis that aim to reduce the tediousness of the process, and the resulting inaccuracy of the results, have been proposed and used in practice. These hybrid techniques do not consist exclusively of full profiles of hypothetical products, as in conventional conjoint analysis, but they start off by asking the respondent a set of self-explication questions (non-conjoint questions that involve no trade-offs), and follow that with partial-profile descriptions. Examples of such techniques include Adaptive Conjoint Analysis and the newer Hierarchical Bayes conjoint analysis.
In Adaptive Conjoint Analysis as implemented by Sawtooth Software (the most frequently used technique for commercial conjoint studies in both the United States and Europe), the survey starts by asking the respondent to eliminate those attribute levels that he or she would find unacceptable under any conditions. Those levels are no longer used in the subsequent part of the interview. Next, the respondent is asked to reduce the levels in each attribute to the 5 levels he or she is most likely to be interested in. The next step in the process asks the respondent to rate the importance of individual attributes; these ratings attempt to eliminate those attributes deemed unimportant, and to generate initial estimates of the respondent's utilities, which subsequently are used to generate a set of customized paired-comparison questions using partial profiles. With each response, the estimates of the respondent's utilities are updated, and appropriate paired-comparison questions generated. These questions are designed to converge and focus on the subspace of attribute comparisons that appears most favored by the respondent based on the earlier responses, with the objective of refining the estimates of that respondent's trade-off profile within that limited subspace.
Clearly, Adaptive Conjoint Analysis relies heavily on the self-explicated evaluation component of the questionnaire, where the decision-maker is asked explicitly to indicate his attitude towards various attributes separately. A key assumption behind that method is that the respondent's attitudes and preferences are pre-existent and stable. Adaptive conjoint relies on that assumption to quickly narrow the choices presented to the interviewee and reduce the workload imposed on him or her. Adaptive conjoint thus precludes the possibility that the respondent might uncover or evolve new personal preferences or attribute trade-off profiles as he or she participates in the study. The problem with that approach is the danger of reification of any preconceived notions or partial, ill-formed preferences the respondent might have a priori, resulting in a sub optimal to the product design problem. In fact, users of Adaptive Conjoint Analysis are warned against allowing respondents to eliminate attribute levels (the first step described in the previous paragraph) “unless there is no other way to make an interview acceptably brief.”
A more recent development, Hierarchical Bayes conjoint analysis, improves on adaptive conjoint through the use of more robust and theoretically more defensible statistical methods. It does not however address the problem described above. Furthermore, Hierarchical Bayes Adaptive Conjoint Analysis relies on the responses of other participants in the study to improve the estimates of each individual's utilities; in other words, Hierarchical Bayes makes it possible to trade the number of the respondents surveyed with the workload on any individual respondent. It is highly computationally intensive procedure however, requiring several hours of running time on a typical personal computer; it is therefore not very useful in a real-time online context. The existing software products perform the Hierarchical Bayes analysis of the data obtained through an adaptive conjoint study after the fact, offline.
The second major shortcoming of conjoint analysis, one that is not addressed by any of the improved methodologies discussed above, stems from the assumption that the different product attributes are independent of one another. Conjoint analysis is a “main effects only” model; it assumes there are no interactions among attributes. In the additive part-worths model that is used universally, an individual's preference for a particular product is assumed to consist of the sum of independent functions of the attribute levels in that product. Using an automotive example again, a consumer's preference for exterior color, bright red versus dark gray for example, is assumed not to depend on body style, whether the automobile in question is a sport coupe or a luxury sedan. Yet we know empirically that bright red is a more popular on sporty cars than it is on luxury sedans. If the researcher suspects that there may be some interaction between two attributes (based on product knowledge or from statistical analysis), the solution within the conjoint analysis framework is to define composite variables (“superattributes”) that are a combination of the two interacting attributes. These super-attributes are given the levels formed by combining the individual attribute levels. Returning to the previous example, the composite attribute would be “color-body style”, and it would take on four levels (two times two): “bright red sports coupe”, “bright red luxury sedan”, “dark gray sport coupe”, and “dark gray luxury sedan”. The problem with that work-around is that it is highly deleterious to the respondent workload. (It is after all the main-effects only aspect of conjoint that makes possible the highly fractionated factorial designs.) Instead of two attributes with two levels each, we now have three attributes with a total of eight levels. This combinatorial explosion is much more severe when a more realistic number of individual attribute levels is used: in the case of five colors and five body styles, we would go from 10 levels (5+5) to a total of 35 levels (5+5+(5×5).) The number of parameters to be estimated by the conjoint study, and therefore the number of questions respondents are subjected to, increase in proportion to the number of these levels.
The “main-effects only” nature of conjoint analysis has a more subtle and insidious effect, as it affects how many marketers and product developers come to think about their products and services. By relying on conjoint analysis to obtain the voice of the customer, they tend to design studies that use those attributes of the product which are more readily decomposable; and they present them in a way that makes it easy for the respondents to separate them. Respondents end up focusing on a few of these attributes, and using them heuristically (as mentioned earlier), and not performing the additional mental processing that would reveal possible interaction between attributes. The result is an artificially good fit to the additive part-worths model, but poor predictive accuracy.
More fundamentally, the very notion that a product or service can be adequately described to a consumer by a set of attribute levels is itself problematic. Since conjoint analysis works by presenting decomposable stimuli to the respondent, it is particularly ill-suited for understanding how consumers evaluate important classes of products, namely, products that are perceived holistically by the consumer. Examples of such “unitary” products include, but are not limited to aesthetic objects, foods, fragrances, and music. In such cases, where the respondent cannot break the stimulus presented to him or her into component parts or attributes, attempting to build simple models of the respondent's preference based on factorially designed studies is unlikely to succeed.
By contrast, this invention does not require that the same factors used by the marketer or designer to alter the product presented to the respondent to assess his or her preference. In the present invention, the respondent is presented with a stimulus that matches the way in which he or she perceives the particular product or service in real-life.